feat: add note storage schema and codec sections to the compiled package - #1309

…compile

cargo-miden is a thin wrapper around the compiler; the note codec orchestration added with the schema work belongs to the compile pipeline, where package post-processing already lives. Owning it in midenc-compile also means every compilation path that builds a note project with a codec pointer produces the `note_codec` section, not only `cargo miden build`.

Move the whole orchestration (manifest pointer lookup, package staging for `from_project!`, sibling codec crate build, component encoding and zero-import validation, section bytes) into `build_project_note_codec` in midenc-compile's cargo module, attach the section from `post_process_package`, and restore `BuildCommand::exec` to its wrapper shape. Runtime dependencies move to midenc-compile; the component-validation dependencies remaining in cargo-miden are dev-only, used by its end-to-end test.

The produced codec section is byte-identical to the previous implementation.

…er review

A multi-pass review of the schema work surfaced correctness holes, trust-boundary gaps, and API footguns. This addresses the branch-scope findings.

Schema emission: a fixed export-name guard symbol enforces the one-`#[note]`-struct-per-crate invariant at link time (two structs previously concatenated into one corrupt section); rendered schemas are resolved with wit-parser during expansion so unsupported shapes fail at the field that causes them; the accepted type surface now matches what the host reader implements; multiline doc comments render as separate WIT doc lines. The tuple-struct and `Vec` rejections are recorded as breaking changes with migration guidance.

Codec builds: the nested build is gated to note targets with focused errors elsewhere, follows the session's profile, toolchain, and target directory, uses the compiler's cargo helpers and JSON artifact paths, stages the in-flight package at a stable path handed to the codec macros through `MIDENC_NOTE_CODEC_PACKAGE_PATH`, and validates the produced component against the pinned world by signatures. Schema and codec sections are attached and read as exactly-one.

Codec execution: each codec call runs in a fresh instance with a fuel budget, a memory limiter, and output-size caps; reported type names can no longer shadow standard codecs or reach outside the note's own schema.

Consumer surface: artifact discovery lives once in miden-note-schema; codec dispatch works for types containing protocol leaves such as `AccountId`; generated string APIs keep a stable shape as schemas gain nested types; `CodecRegistry::empty`/`with_standard_codecs` replace the ambiguous constructor pair; `miden-note-bindings` is a facade that supplies generated-code dependencies and stays hygienic across multiple expansions, backed by an internal `crate_path` override in the felt-repr derives; embedded core-type definitions are structurally verified before native mappings apply; end-to-end tests exercise the in-process pipeline and inherit workspace patches.

…lishing

The branch's first CI run flagged four problems: dependencies declared inline in several crates instead of the workspace table, a wasmtime feature set that does not compile under the release job's feature unification, unused dependency declarations, and workspace packages missing from the release classification. Separately, the note crates' long-term home changed: they stay in this repository instead of moving to the protocol repo, so they publish with the sdk release unit.

Promote `prettyplease`, `wit-bindgen`, and `proc-macro-crate` to workspace dependencies. Add wasmtime's `std` feature, which its mmap-backed runtime needs on Linux. Remove workspace entries nothing inherits and a leftover dev-dependency, and move the bindings expansion goldens next to the tests that read them.

Classify all six note crates as published members of the sdk unit (`version-source = "sdk"`) at the sdk train version, and update the changelog wording accordingly.

Two release-workflow jobs stayed red after the previous hygiene pass, each with its own root cause.

The per-member check job compiled the host-side note crates for wasm: `sdk/.cargo/config.toml` applied `target = "wasm32-wasip1"` to every cargo invocation inside an sdk crate directory, which sent wasmtime onto its no-std custom platform and broke the build. The directory-wide config is replaced by verbatim per-crate configs in the ten guest-side sdk crates, so guest builds behave exactly as before while host crates build for the host.

The package-closure job could not build `midenc-compile` from a registry: its pinned copy of the note-codec world WIT was an `include_str!` that escaped the package root, and the codec macros crate carried the same latent escape. Each embedding package now ships its own copy of the file, and a test in the unpublished integration crate locks all three copies together. This avoids the alternative of making the compiler depend on `miden-note-codec`, which would pull the author-side codec stack into the compiler's dependency tree for the sake of one string.

…ebase

The rebase onto the note-constructor and typed-script-args work left two loose ends: a rustfmt reflow of the merged import block in the note macro was lost while relocating fixes into their commits, and the new upstream constructor test pins cycle counts that shift by the schema uniqueness guard's fixed advice-map cost.

Restore the formatting and refresh the constructor test's cycle expectations; the +10-cycle delta matches the guard cost already reflected in the other mockchain expectations.

…ema review

A second multi-pass review of the schema work surfaced silent type-identity holes, resource-exhaustion paths on untrusted input, a publishability gap, and infrastructure drift between the macro crates.

Type identity: registering two exported types with the same name but different shapes is now an error, as is a note field whose unregistered type name merely collides with an SDK core type — both previously produced schemas that disagreed with the guest encoding silently.

Untrusted input: the schema model is a memoized graph with explicit limits on section size, type count, nesting depth, and root width, so a small adversarial WIT document can no longer explode into exponential allocation; codec components are size-capped before compilation and their tables are bounded at instantiation.

Packaging: the canonical note-codec world WIT moves into the new dependency-free `miden-note-codec-wit` crate, replacing three held-together copies; `miden-note-schema` keeps the one package-local copy its wasmtime bindings require, equality-tested against the crate. The `codec-component` feature now builds from a registry. The changelog documents the author-codec surface and the `note-codec-crate` manifest key, and a codec declaration applies per note target instead of failing sibling targets; core-module builds reject schema sections with guidance instead of dropping them.

Infrastructure: note artifact discovery follows the same identity-ordered policy as FPI dependency resolution and the in-flight package stages through the package cache, retiring the bespoke env-var handoff; the nested codec build stages immutably and honors the outer `--locked`/`--offline` policy; generated runtime paths are code-generation inputs, deleting both post-hoc AST rewriters, with one felt-repr redirection mechanism, dependency-rename-safe paths, and a one-schema-per-crate contract with actionable errors. The three walkers share one standard-leaf definition.

The codec crate previously compiled for wasm32-unknown-unknown and a separate ComponentEncoder pass wrapped the module into a component. Building for wasm32-wasip2 lets rustc link the component directly through wasm-component-ld, removing the encoding step and its failure modes.

The wasip2 standard library wires WASI interfaces into every component, so the zero-import property is replaced by an equivalent guarantee: build validation accepts `wasi:*` imports only, and consumers stub every import as a trapping function at instantiation, so codec code can still reach no host capability. The mockchain end-to-end test confirms a stubbed component instantiates and converts values normally.

The component-export tests build through the same path as the compiler, and the note-schema crate drops its now-unused wit-component dev-dependency.

The component and consumer test fixtures are standalone crates in
temporary directories, so their offline cargo builds resolved
dependencies from scratch against the local registry index. Any
release published after the workspace lockfile was last updated made
the fresh resolution select a version that was never downloaded, and
the offline build failed. A yanked release breaks such builds the
same way.

Seed each fixture with the workspace Cargo.lock before the build so
resolution reuses the exact versions the workspace build already
fetched into the cargo cache.

The schema renderer resolved note field types by their last path
segment, so a foreign type that only shared the registered name could
publish a schema whose shape differs from the guest encoding.
#[export_type] now stamps each exported type with a hidden structural
shape constant, and both export sites and #[note] sites emit
compile-time checks that read the constant through the type path as
written. A name-only match no longer compiles.

The note site now emits SDK core-type identity guards only for the
note struct's own fields. Registry definitions keep the guards at
their #[export_type] sites, so a note that refers to exported types
from other modules compiles.

Both proc-macro registries are now keyed by the expanding crate and
replace a re-registration from the same source location. Long-lived
macro hosts such as the rust-analyzer proc-macro server no longer
accumulate stale registrations that surface as phantom conflicts, and
real conflict errors tell the user to restart the macro server when
the error appears in an IDE.

The macOS link-section name is now derived from the canonical section
name constant instead of a hand-truncated copy.

The compiler attached note storage schemas without applying the limits
every consumer enforces on read, so an oversized schema failed at the
consumer instead of the producer build. Package assembly now validates
the schema with the consumer implementation before it attaches the
section, and all section attachers share the one duplicate-rejecting
policy.

The nested codec build keyed its Cargo target directory by the staged
package content, which rebuilt the whole codec dependency graph on
every note change and never removed old directories. The target
directory is now shared; the staged-cache path is recorded in
dep-info, so a key change still re-expands the codec macros. Stale
staged packages are removed after seven days, offline sessions no
longer spawn a network-capable rustup install, and custom Miden
profiles map to the Cargo dev profile that the independent codec
workspace actually defines.

The note section identifiers, the 16-byte link-section padding, and
the trailing-NUL stripping were each spelled at several call sites
with small differences. wasm-metadata now owns section-id accessors
and the padding helpers, and every producer and consumer uses them.

Package discovery in miden-note-schema now follows the documented
package-cache contract: an empty MIDENC_PACKAGE_CACHE value counts as
unset, the shared environment constant and file-name helper replace
local copies, and a cache miss names the searched directory and the
expected file names instead of a wrong fallback path.

The two p2id end-to-end tests build the same example projects in
place and rewrite the same package file, and the release-profile CI
job runs them concurrently from two test binaries. Each test now
holds a shared advisory file lock in the workspace target directory
for the whole build-write-consume span. The unit-job exclusion in CI
now says why these tests run in the release-profile job.

The #[note] and #[export_type] reference documentation now covers the
emitted WIT schema section, the supported field surface, the
define-before-use ordering, the written-type identity checks, and the
one-note-per-crate guard. The migration guide explains the new
restriction, and the codec and bindings crates carry usage examples
with the required macro ordering. The SDK changelog lost the sections
that a rebase auto-merge had duplicated, and the note entries now
reference their issue.

The implementation-plan and follow-up-issue drafts are working
documents and leave the repository. The workspace manifest dropped a
Cargo exclusion for a directory that no longer exists, the fanned-out
guest .cargo configs now say why each crate carries its own copy, and
the frontend lost a dead schema-rejection function that contradicted
the shipped core-module propagation behavior.

Enabling the proc-macro2 span-locations feature for the macro
registries grows Span and with it syn::Type, which pushed the
SymbolType enum over the clippy large-enum-variant threshold. The
concrete variant now boxes the type.

Note codecs now always build with the Cargo release profile - a dev
wasm32-wasip2 cdylib carries debug info far past the consumer size
limit - and the compiler enforces the shared component size limit
before it attaches the codec section, so a package that builds is a
package that consumers accept.

The staged-package probe now uses the package-cache file-name helper,
so dotted package names hit the same file the writer produces and the
content-identity guard covers them.

The storage builder encodes a codec-registered record that was
assembled from child-path values as one subtree and runs the codec
validation over it, so the builder can no longer produce storage that
the same registry rejects on decode.

Both macro crates emit fully qualified rebuild-tracking constants and
take the cache environment name from the shared constant. The codec
world check derives the expected identity from the pinned WIT instead
of literals. Memoized schema nodes record their subtree depth so
reuse cannot exceed the schema depth bound. Dead code, duplicated
resolution-policy arguments, and diagnostic literals are cleaned up,
and the tricky spots called out by review carry explanatory comments.

The p2id build lock, the example compilation helper, the workspace
root helper, and the wasm-target probe existed as verbatim copies
across four crates. They now live in the integration test-support
crate, and the component fixture no longer embeds its schema text as
a second literal.

… extraction

A failed nested codec build ended the whole process with Cargo's
status, so the recovery guidance for locked and offline builds never
reached the user and in-process callers lost their test binary. The
codec path now uses a returning Cargo runner; the frontend build
paths keep their exit semantics.

Staged note-package entries never refreshed their age on reuse, so
the seven-day collector of one build could remove an entry another
build was actively using. A cache hit now touches the entry, and the
collector also removes stray files.

The core-types interface extraction balanced braces by raw character
count; a brace inside a WIT comment corrupted every emitted schema.
The scan now tracks line and block comments while copying the text
verbatim.

The nested-build environment scrub also removes the Cargo rustflags
variants, the unreachable alias arm and the dead tuple-note encoding
arm are gone, the wit-bindgen pins moved to workspace dependencies
with the wit-parser skew rationale, host-side per-crate Cargo configs
carry accurate comments, and the load-bearing spots named by review
(the package-cache handshake, the nested target directory, the codec
attach ordering) and the remaining items without documentation now
carry it.

Two more test sites build the same example projects in the same CI
lane as the locked pair; all four now hold the shared advisory lock,
renamed to match its wider scope. The note-codec crate returns to a
local wasm-target probe instead of pulling the compiler graph through
the integration-support crate for one helper. The dex codec test
restores its environment through a drop guard, so a panic cannot leak
a removed variable into other tests, and the duplicated example
compilation helper is gone.

Builtin names were trusted by their last path segment, so a foreign
or shadowing type named Option, Result, or a primitive could change
the encoded layout while the schema still declared the builtin. Every
builtin reference now carries the same nominal identity check that
pins SDK core types, proven against the ::core definitions.

midenc-compile declares no_std with std-only dependencies behind the
std feature, but the note-schema and sha2 dependencies were added
unconditionally; both are optional now and the crate builds again
without default features. Offline target detection probes the sysroot
first and only gates the rustup install step, so linked toolchains
work offline. Nested codec build failures always name the codec
manifest, and the lockfile and network advice is phrased for the
failures it can actually explain. Staged-cache freshness works
through file mtimes, so the age refresh is portable, and the
base-macros registry tests serialize on a shared lock. The macro
registry docs no longer promise a declaration order the FQN-keyed
map does not keep.

The example build lock moved from individual test sites into the
shared compilation helpers, so every test that builds an example
project holds it, including the sites the previous pass missed, and
re-entrant double locking cannot happen. Test-side package files are
written to a temporary name and renamed into place, so a concurrent
reader never sees a partial file. The dex codec test, which builds
its example in place, takes the lock directly, and the test-side
nested builds scrub the same environment variables as the production
sites.

The migration guide now covers the duplicate-registration error for
different Rust types with identical shapes. The per-crate Cargo
config comments describe what the files actually set. The workspace
comment on the wit-parser version skew states the direction the
lockfile shows. The seven note crates move from the never-published
section of the release configuration into the sdk unit they belong
to, and the note-codec crate description names the canonical-value
rule correctly.

…iguous

The builtin identity guard rendered a unit Result argument as empty
text, so Result<(), T> and Result<T, ()> fields in exported types
failed to expand with an internal reconstruction error. The unit
placeholder now renders as () and the supported shapes compile again.

A duplicate #[export_type] registration was treated as benign by name
and shape alone, so two different same-named types registered
silently while the migration guide promised an error. The benign path
now also requires the same expansion location; a second item with the
shared name conflicts regardless of shape.

The environment variables that must not leak into nested Cargo builds
are one shared constant in wasm-metadata, used by every scrub site.
The duplicated Cargo-argument builder in the Rust frontend collapsed
into one function. The dead commented-out declaration left the
embedded core-types WIT, which shrinks the schema section of every
note package, and the identity-guard entry point documents both
checks it emits.

Two builders of the shared examples ran without the example build
lock, so the serialization it promises did not hold; both take it
now. The atomic package writer in the test-support crate delegates to
the production writer, which keeps dotted package names intact. The
wasm-target probes follow the sysroot-first rule production adopted,
so linked toolchains run the gated tests instead of skipping them,
and the core-types golden text exists once per expectation file.

All ten fanned-out config copies carry one comment that is accurate
for guest, host, and proc-macro crates alike.

The atomic package publisher now reports through anyhow, so the
test-support wrapper converts its error into the io error its callers
expect. The #[note] expansion now also implements the SDK ActiveNote
trait, so the standalone unit-note test provides the same trait
stand-in as its sibling. The lockfile records the dependency edges of
the merged manifests.

…uilds

The three nested Cargo builds in the note tests (the generated
bindings consumer, the codec fixture, and the schema fixture) now
share the hash-keyed build directory that the test-support crate
already uses, so they stop rebuilding the native Miden dependency
cone into private directories. The generated consumer project keeps
only file and line debug information; it exists to prove that the
bindings compile and run, and full DWARF for that dependency cone
costs gigabytes in the shared build directory.

…igration

…nifest table

The codec build read the session for its work directory, the profile, and the cargo policy flags. A package post-processor cannot borrow the session, so the build now derives everything from the codec crate: its work directory is `<codec-crate>/target/midenc.note-codec/`, with the same content-addressed staging, nested cargo target, and age-based GC, and the nested build no longer inherits `--locked` and `--offline`. `post_process_package` loses its session parameter, of which the codec attach was the only consumer.

The manifest table becomes `[package.metadata.midenc.note-codec]` with a `crate` key, grouped under the `midenc` namespace like the VM's event-handler plugin, and the reader rejects unknown keys and non-table values. This pre-adopts the shape of the future `PackagePostProcessor` plugin crate, so the move at the VM 0.31 migration is code motion only.

The codec consumer ran untrusted components under the engine's default feature set, with private limit constants, no bound on the work that compilation and instantiation cost before fuel applies, and one undifferentiated error for every call failure. This ports the policy ideas from the VM's Wasm event-handler runner onto the wasmtime consumer.

Every Wasm proposal is now set by name: the wasip2 defaults stay on, floats stay on with NaN canonicalization, and everything else is off. A new feature-free structural validator caps functions, globals, tables, memories, segments, imports, exports, and signature widths per core module, the number and nesting of core modules, and the average function size, with the numbers wasmi's strict limits use; the producer applies the same function at build time, so a component every consumer would refuse fails the author's build. Runtime limits move into a `CodecLimits` struct with today's values as defaults and a `load_from_package_with_limits` entry point, documented as host policy that a package cannot set. Call failures carry a `CodecFailure` class, with limit hits recorded by a resource-limiter wrapper instead of parsed from engine messages. The nested codec build pins `RUSTFLAGS` to disable SIMD after scrubbing inherited flags, so a codec crate's own cargo config cannot enable a feature every consumer rejects.

…plies

The structural check counted core-module sections but not component-level ones. wasmtime's inliner expands every nested component instantiation at compile time, so a tiny component could force billions of initializers before any store limit or fuel applied. Core and component instantiations are now budgeted across the whole component tree, every other component-level section is capped, a component start function is rejected, and defined tables and memories are counted against the store's own limits.

The Wasm feature policy was implicit: threads and GC were off only because of the wasmtime cargo features this crate selects, which feature unification in a host can flip back on, and the producer never checked features at all. The policy is now an explicit wasmparser feature set validated payload by payload, function bodies included, inside one shared entry point that both the producer's build check and every consumer's load run.

Discovery lifted the reported type list eagerly through the generated bindings, so string descriptors aliasing one small region could force unbounded host allocation at load. The list is now read through a typed function with lazy cursors, the length checked before any element is read, and each name capped before it is copied.

Failures during instantiation and discovery, and every host-side cap, now report their `CodecFailure` class. The limits documentation states the per-memory and per-table semantics, and the fixed store counts alias the structural budgets. A package without a codec section loads the base registry instead of failing. The manifest check precedes staging, the test fixture pins the guest rustflags like the real build, and `tempfile` becomes a dev-dependency.

… names

A chain of empty records doubled forty times passes every schema limit with zero felts, yet decoding, builder validation, and code generation expand its tree exponentially. Every resolved type now carries its expanded node count next to its layout, and resolution rejects a type above a fixed budget, so one check protects every traversal on both the producer and the consumer. The native felt-repr check in the code generator is memoized by type identity.

A schema failure in `#[note]` discarded the struct, so every use site reported a missing type and hid the real diagnostic. The struct is now emitted next to the error.

Generated bindings and codec dispatch code used unqualified prelude names, so a WIT record named `vec`, `string`, or `option` shadowed them inside the generated module. Every emitted prelude type is now fully qualified, and a binding test compiles such a schema.

The template skill still advertised `Vec` fields for `#[note]` structs, which the macro rejects; it now states the named-field-or-unit rule and the one-`#[note]`-per-crate rule, and the embedded template bundle is regenerated. The SDK changelog gains the one-note-per-crate breaking-change entry the migration guide documents. The codec crate docs show the manifest opt-in. The test-only registry reset is documented. Comments about `--locked` and `--offline` no longer claim every nested build inherits them. A dead sort of already-unique package paths is removed.

…date

The parent branch changed the SDK allocator and heap growth, which moved every note-script cycle count and note-package size by a constant. The expectations are re-measured; the branch's own deltas over the parent are unchanged.

…n it exists

The consumer-crate test required a `[patch.crates-io]` table in the workspace manifest and failed once the workspace dropped its commented-out table. The copy is now optional: a workspace without patches gives the consumer crate no patches, which is the correct mirror.

…maining sandbox gaps

The structural check capped component-level sections one section at a time, but a component may repeat a section, so the caps bounded nothing. Item counts are now accumulated across the whole tree per kind, core type sections gain a cap, and the module documentation states exactly which rules apply, including that only a component-level start function is rejected.

The budgets also counted declarations while the store counts runtime objects: a module instantiated twice creates two memories, and a nested component multiplies whatever it creates. Each component frame now keeps its module and component index spaces, an imported or aliased entry is not instantiable, and every core or component instantiation adds what it creates, so a component that passes the check instantiates inside the store limits. Every structural rejection reports the limit class, load and lift failures report the trap class, and a store-count rejection at instantiation is classified as a limit.

Loading a package without a codec section no longer requires a schema section first, so unit-note packages get the standard registry. Generated code fully qualifies its trait bounds and derives, the codec registry keys custom types by their WIT name with the generator's identifier rule so a reserved name resolves and a same-name collision is reported, and the written-type identity records a leading path separator. The artifact resolver reports the manifests it read so generated bindings track a package rename. Generated bindings parse the schema once, the export test pins the guest flags and validates its component, and several fields and helpers gain documentation.

…e-size build

A schema error in `#[note]` dropped every generated impl with the schema static, so a real note crate reported trait-bound errors from its `#[note] impl` block next to the schema diagnostic. The error now replaces the schema static only; the felt-repr impls, the `ActiveNote` impl, and the uniqueness guard are still emitted, and a rustc-driven test with a note-script body proves the diagnostic stands alone.

Two of the four example builds in the package-size test bypassed the shared build lock that the network suite holds for the same examples. All four now build through the locking helper. The remaining hand-rolled nested-cargo environment scrubs in the integration tests use the shared helper. The codec WIT crate becomes an optional dependency of the compiler behind its `std` feature, and the script-arguments crate gains the per-crate cargo config its siblings carry so the per-member check targets Wasm.

…de changes

The template skill claimed a custom field type only needs the felt-repr derive and forbade `Asset` and `Word` as note fields. A custom type must carry `#[export_type]` and be declared before the `#[note]` struct, and the SDK core records that implement the felt-repr traits are accepted. The guidance now lists the supported surface, and the embedded template bundle is regenerated.

The compiler changelog gains the `--locked` and `--offline` flags, the post-assembly note codec build, and the note storage schema section. The one-`#[note]`-per-crate rule is stated as one per linked artifact in the migration guide, the SDK changelog, and the macro documentation, since a note crate cannot depend on another note crate.

The changelogs are generated from the commit messages at release time, so the entries this branch accumulated in the compiler and SDK changelogs are removed and both files match the parent branch again.